Supplementary material: Learning and Selecting Features viaPoint-wise Gated Boltzmann Machines

There are many classification tasks where we are given a large number of unlabeled examples in addition to only a few labeled training examples. For such scenario, it is important to include unlabeled examples during the training to generalize well to the unseen data, and thus avoid overfitting. Larochelle and Bengio (2008) proposed the semi-supervised training of the discriminative restricted Boltzmann machine by combining the generative objective defined on the unlabeled examples with the discriminative objective. Similarly to their approach, the supervised PGBM can be trained in a semi-supervised learning framework. Specifically, we can use the input data log-likelihood defined on the unlabeled data as a regularizer.